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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments with respect to claims 1-28 and 31-35 have been 
considered but are moot in view of the new ground(s) of rejection. 

Specification 

The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) because they 
include the following reference sign(s) not mentioned in the description: all reference 
numbers (100-141) in figures 5-7. A proposed drawing correction, corrected drawings, 
or amendment to the specification to add the reference sign(s) in the description, are 
required in reply to the Office action to avoid abandonment of the application. The 
objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1 and 31-33 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent 5,559,999 to Maturi et al. (Maturi) in view of U.S. Patent 5,521 ,927 to 
Kim et al. (Kim) and U.S. Patent 6,357,042 to Srinivasan et al. (Srinivasan). 
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Regarding claim 1, Maturi teaches instigating an event such as synchronizing the 
audio with the video (Abstract). Maturi teaches a system time clock (STC) located and 
provided in the decoder, a system clock reference (SCR) as used in MPEG-I or 
Program Clock Reference (PCR) as used in MPEG-I I which is the clock reference given 
by the encoder (col. 2, II. 14-22). Along with the time clocks, Maturi teaches a 
Presentation Time Stamp (PTS) which indicates the proper presentation of data (col. 1, 
II. 62-65). Maturi teaches an audio/video decoder as shown in figure 1 , which generates 
a reconstructed application includes time to initiate the event (col. 7, II. 28-36). The time 
to initiate the event (claimed application time moment) is a function of STC (claimed 
current moment of a system time), a PTS (claimed sampled application time moment), 
and decoding time stamp (DTS) which is associated with a sampled moment of a 
system time associated with sample application time moment. Further, Maturi teaches 
performing the action once the time is less than one frame time interval different (col. 7, 
II. 37-40). 

Maturi teaches an application time moment as a function of STC, however, 
Maturi is silent on the moment being based in part upon a delay in the packetization 
process. In analogous art, Kim teaches adding an offset corresponding to the interval 
between a coding start time point in the encoder and a time point that a stamp is coded 
in the transport stream (col. 4, II. 17-24), which reads on a delay in the packetization 
process. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Maturi by using an offset based in part upon a 
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delay in the packetization process as taught by Kim in order to enable precise 
synchronization among streams. 

Maturi teaches the events being audio and video, but is silent on an event apart 
from reconstructing video and audio. In analogous art, Srinivasan teaches integrating 
an event, such as host spots and interactive regions (col. 37, II. 3-6) in order to present 
metadata during a frame or series of frames (col. 37, II. 35-41 ). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Maturi by synchronizing events another than audio and video as taught by 
Srinivasan in order to enhance the main video stream an incorporating metadata 
thereby increasing interactivity and enabling the user to select and review portions of 
video presentations (Srinivasan: col. 3, II. 8-23, col. 3, II. 42-52). 

Regarding claim 31, Maturi teaches the SCR and PCD (col. 3, II. 31-35) as a 
system time. Maturi teaches the clock as a counter (col. 7, II. 22-27). Maturi teaches 
the STC derived from SCR or PCR (col. 3, II. 31-43), which is divided to reduce the 
resolution. Maturi teaches a comparator (fig. 10, label 42, fig. 1 1 , label 42), and the pre- 
parer (22) and post-parser (24) clearly have registers in order to process the audio 
decoding in less than one frame time interval different from the counter (col. 7, II. 37- 
40). As described previously in the rejection, there exist PTS and presentation units 
(PU) which are not necessarily sent together, which equate to the first, second, and 
third data unit presentation moments (col. 1-2, II. 66-4), which are clearly depacketized 
in order to retrieve the data. The system of Maturi provides the time recovery unit and 
the presentation units (fig. 3). 
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Maturi teaches an application time moment as a function of STC, however, 
Maturi is silent on the moment being based in part upon a delay in the packetization 
process. In analogous art, Kim teaches adding an offset corresponding to the interval 
between a coding start time point in the encoder and a time point that a stamp is coded 
in the transport stream (col. 4, II. 17-24), which reads on a delay in the packetization 
process. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Maturi by using an offset based in part upon a 
delay in the packetization process as taught by Kim in order to enable precise 
synchronization among streams. 

Maturi teaches the events being audio and video, but is silent on an event apart 
from reconstructing video and audio. In analogous art, Srinivasan teaches integrating 
an event, such as host spots and interactive regions (col. 37, II. 3-6) in order to present 
metadata during a frame or series of frames (col. 37, II. 35-41). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Maturi by synchronizing events another than audio and video as taught by 
Srinivasan in order to enhance the main video stream an incorporating metadata 
thereby increasing interactivity and enabling the user to select and review portions of 
video presentations (Srinivasan: col. 3, II. 8-23, col. 3, II. 42-52). 

Regarding claim 32, the combination of Maturi and Srinivasan teaches a 
descriptor of an event. 

Regarding claim 33, Maturi teaches receiving a payload of the data unit 
comprising application time sample, which is clearly generated via the encoder and is 
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sent downstream (col. 1 , II. 45-54). Maturi is silent on a data server. Official Notice is 
taken that a data server constructing payloads is well known in the art. Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to modify Maturi by implementing a data server in order to create the data that is 
sent downstream thereby providing the proper PTS information to properly synchronize 
events with the video. Further, Maturi teaches a multiplexer adding a heading 
comprising said second data unit presentation time (col. 1-2, II. 66-4, col. 2, II. 55-59, 
col. 4, II. 40-47). 

4. Claims 2-17, 27, and 28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 5,559,999 to Maturi et al. (Maturi) in view of U.S. Patent 
5,521,927 to Kim et al. (Kim). 

Regarding claim 2, Maturi teaches instigating an event such as synchronizing the 
audio with the video (Abstract). Maturi teaches a system time clock (STC) located and 
provided in the decoder, a system clock reference (SCR) as used in MPEG-I or 
Program Clock Reference (PCR) as used in MPEG-II which is the clock reference given 
by the encoder (col. 2, II. 14-22). Along with the time clocks, Maturi teaches a 
Presentation Time Stamp (PTS) which indicates the proper presentation of data (col. 1 , 
II. 62-65). Maturi teaches an audio/video decoder as shown in figure 1 , which generates 
a reconstructed application includes time to initiate the event (col. 7, II. 28-36). The time 
to initiate the event (claimed application time moment) is a function of STC (claimed 
current moment of a system time), a PTS (claimed sampled application time moment), 
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and decoding time stamp (DTS) which is associated with a sampled moment of a 
system time associated with sample application time moment. Maturi teaches 
associating the event and instant with a correlating application time, where the instant is 
less than one frame time interval different (col. 7, II. 37-40) and application time is 
correlated with a presentation moment of system time as discussed above. Maturi 
teaches performing the action once the time is less than one frame time interval 
different (col. 7, II. 37-40). 

Maturi teaches an application time moment as a function of STC, however, 
Maturi is silent on the moment being based in part upon a delay in the packetization 
process. In analogous art, Kim teaches adding an offset corresponding to the interval 
between a coding start time point in the encoder and a time point that a stamp is coded 
in the transport stream (col. 4, II. 17-24), which reads on a delay in the packetization 
process. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Maturi by using an offset based in part upon a 
delay in the packetization process as taught by Kim in order to enable precise 
synchronization among streams. 

Regarding claim 3, Maturi teaches a Decoding Time Stamp (DTS) which 
incorporating a sample moment of the application time in a data unit to be presented at 
the presentation moment approximately corresponding to sample moment of the system 
time by taking the difference of the fixed decoding time from the PTS (col. 7, II. 34-36). 
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Regarding claim 4, Maturi teaches incorporating the instant at approximately said 
correlating time, where the data unit is to be presented at the presentation moment (col. 
7, II. 34-40). 

Regarding claim 5, Maturi teaches presenting the event at the presentation 
moment at the instant of the presentation moment, when the instant is less than one 
frame interval different (col. 7, II. 40-37). 

Regarding claim 6, Maturi teaches synchronizing the audio (claimed event) to the 
video elements (col. 3, II. 25-30). Maturi teaches associating a Presentation Time 
Stamp (PTS) with a System Clock Reference (SCR) or a Program Clock Reference 
(PCR) (col. 2, II. 13-21), which equates to associating a sample application time 
moment with a sample system time moment. Further, Maturi teaches using the PTS 
and SCR/PCR to synchronize the reconstructed local clock, which is the System Time 
Clock (STC), which as taught by Maturi manages both the audio and video and 
consequently reads on both the claimed application and system clock (col. 2, II. 13-21, 
col. 3, II. 21-24). Maturi teaches associating the audio (claimed event) and video 
(claimed instant of said program element) with a PTS (col. 7, II. 22-27). Maturi teaches 
relating the application time and presentation moment by creating a Decoder Time 
Stamp (DTS), which is the PTS minus a fixed decoding time (col. 7, II. 34-36). Maturi 
teaches instigating the audio at a time moment generated by the STC corresponding to 
the DTS (col. 7, II. 34-45). 

Maturi teaches an application time moment as a function of STC, however, 
Maturi is silent on the moment being based in part upon a delay in the packetization 



Application/Control Number: 09/492,561 Page 9 

Art Unit: 261 1 

process. In analogous art, Kim teaches adding an offset corresponding to the interval 
between a coding start time point in the encoder and a time point that a stamp is coded 
in the transport stream (col. 4, II. 17-24), which reads on a delay in the packetization 
process. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Maturi by using an offset based in part upon a 
delay in the packetization process as taught by Kim in order to enable precise 
synchronization among streams. * 

Regarding claim 7, the system of Maturi uses the same STC for the application 
time clock and system time clock, accordingly, the sample application time moment is in 
the same data unit and is presented at the sample system time. 

Regarding claim 8, Maturi teaches selecting a moment from an application 
timeline generated by an application clock, where the audio PTS is sent downstream in 
the program stream (col. 5, II. 3-10). As discussed in claim 6, Maturi teaches 
accounting for the time it takes to shift decode the audio and provides a DTS and 
consequently incorporates this difference in order to timestamp the PTS for the proper 
synchronization (col. 7, II. 34-40). 

Regarding claim 9, Maturi teaches comparing the STC (claimed current time 
moment) to the PCR/SCR time moment in order to synchronize the overall system 
clocks at the encoder and decoder (col. 7, II. 22-27). As discussed in claim 7, clearly 
the application clock and STC are set equal to each other at all times, which reads on 
when said current system time moment corresponds to said same system time moment. 
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Regarding claim 10, Maturi teaches creating a DTS, which is the PTS minus a 
fixed decoding time, which permits the audio to be synchronized to the video to within a 
frame (col. 7, II. 34-40). 

Regarding claim 1 1 , Maturi teaches presenting the data within a frame of the 
video (col. 7, II. 34-40), which reads on said instant at said presentation moment of said 
system time. 

Regarding claim 12, the limitations of claim 12 have been addressed in the 
discussion of claim 6, and the claim 12 adds the limitations of a second data unit, which 
is also taught by Maturi. Maturi teaches that PTS and access units are carried in 
different layers of the hierarchy and are not necessary sent together. Consequently, 
Maturi teaches transmitting access units to the receiver in a second data unit having a 
second presentation moment of system time associated with said correlating moment of 
the application time (col. 1-2, II. 66-4, col. 2, II. 22-35). 

Regarding claim 13, the limitations of claim 13 have been addressed in the 
discussion of claims 6, 7, and 12. 

Regarding claim 14, Maturi teaches comparing the STC (claimed current time 
moment) to the PCR/SCR time moment in order to synchronize the overall system 
clocks at the encoder and decoder (col. 7, II. 22-27). As discussed in claim 7, clearly 
the application clock and STC are set equal to each other at all times, which reads on 
when said current system time moment corresponds to said same system time moment. 
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Regarding claim 15, Maturi teaches encoding the PTS, for indicating the 
presentation time, where the PTS is the time stamp associated with an access unit, 
which is all sent to the receiver (col. 1-2, II. 66-4). 

Regarding claim 16, in the system of Maturi, video and audio data is constantly 
streaming downstream to the receiver; accordingly Maturi teaches receiving a third data 
unit and transmitting the third data unit to the receiver in a similar fashion as described 
for the second data unit. 

Regarding claim 17, as discussed above with the relationship between access 
units and PTS, a third data unit can have a temporal relation to the first data unit 
presentation unit. 

Regarding claim 27, Maturi teaches generating a system and application time 
(see discussion of claim 9) and selecting a PTS (claimed sample moment of the 
application time) at the encoder (col. 1 , II. 62-65) and sending the data downstream to 
the user in a first data unit (col. 1-2, II. 66-4). Further, Maturi teaches generating a DTS, 
synchronized to the system time by the sample moment and first data unit presentation 
moment (col. 7, II. 34-37), further the examiner notes that by determining the DTS, the 
DTS is associated with the audio even and said instant of said video with a correlating 
moment of said application time, such as DTS = PTS - fixed decoding time; Maturi 
teaches presenting the system time in order to play the audio event synchronized with 
the video (col. 7, II. 22-40). In addition, fig 3 has a pre-parser 22 and a post-parser 24. 

Maturi teaches an application time moment as a function of STC, however, 
Maturi is silent on the moment being based in part upon a delay in the packetization 
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process. In analogous art, Kim teaches adding an offset corresponding to the interval 
between a coding start time point in the encoder and a time point that a stamp is coded 
in the transport stream (col. 4, II. 17-24), which reads on a delay in the packetization 
process. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Maturi by using an offset based in part upon a 
delay in the packetization process as taught by Kim in order to enable precise 
synchronization among streams. 

Regarding claim 28, Maturi teaches a clock to reconstruct the event as shown in 
fig. 10, 1 1 , see also the discussion of claim 27. 

5. Claims 18-26, 34, and 35 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 5,559,999 to Maturi et al. (Maturi) in view of U.S. Patent 
6,357,042 to Srinivasan et al. (Srinivasan); 

Regarding claim 18, Maturi teaches generating a system and application time 
(see discussion of claim 9) and selecting a PTS (claimed sample moment of the 
application time) at the encoder (col. 1, II. 62-65) and sending the data downstream to 
the user in a first data unit (col. 1-2, II. 66-4). Further, Maturi teaches generating a DTS, 
synchronized to the system time by the sample moment and first data unit presentation 
moment (col. 7, II. 34-37), further the examiner notes that by determining the DTS, the 
DTS is associated with the audio even and said instant of said video with a correlating 
moment of said application time, such as DTS = PTS - fixed decoding time; Maturi 
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teaches presenting the system time in order to play the audio event synchronized with 
the video (col. 7, II. 22-40). 

Maturi teaches the events being audio and video, but is silent on an event apart 
from reconstructing video and audio. In analogous art, Srinivasan teaches integrating 
an event, such as host spots and interactive regions (col. 37, II. 3-6) in order to present 
metadata during a frame or series of frames (col. 37, II. 35-41). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Maturi by synchronizing events another than audio and video as taught by 
Srinivasan in order to enhance the main video stream an incorporating metadata 
thereby increasing interactivity and enabling the user to select and review portions of 
video presentations (Srinivasan: col. 3, II. 8-23, col. 3, II. 42-52). 

Regarding claim 19, the limitations of claim 19 have been addressed in the 
discussion of claim 14; claim 19 further adds the limitation of incrementing the 
reconstructed application time. Maturi teaches a STC which is incremented at 90 kHz 
(col. 7, II. 22-27). 

Regarding claims 20, Maturi teaches the SCR and PCD (col. 3, II. 31-35) as a 
system time signal having a system time unit frequency. Maturi is silent on dividing the 
signal to produce a trigger signal having an application time unit frequency, while 
teaching the application time frequency, STC (col. 21-24). Official Notice is taken that 
dividing a clock is well known in the art. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify Maturi to divide 
the clock in order to derive another lower resolution clock in order to synchronize 
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playback thereby creating a more robust system. Maturi teaches the clock as a counter 
(col. 7, II. 22-27). 

Regarding claim 21, Maturi teaches a DTS, which is adjusted for a period 
required to create, said first data unit (col. 7, II. 34-36). 

Regarding claim 22, in the system of Maturi, video and audio data is constantly 
streaming downstream to the receiver; accordingly Maturi teaches receiving a second 
data unit and timestamping the second data unit to the receiver in a similar fashion as 
described for the first data unit. 

Regarding claim 23, in the system of Maturi, video and audio data is constantly 
streaming downstream to the receiver; accordingly Maturi correlating a second data unit 
and transmitting the second data unit to the receiver in a similar fashion as described for 
the first data unit. 

Regarding claim 24, clearly in a streaming system, the data unit is transmitted to 
the receiver prior to said reconstructed application time corresponding to said 
correlating moment in order to properly process the audio (col. 7, II. 37-53). 

Regarding claims 25 and 26, Maturi teaches continuously streaming audio and 
video, wherein the system has a temporal relationship to the first data unit presentation 
moment in order to re-create audio and video in the proper sequence. 

Regarding claims 34 and 35, Maturi teaches the SCR and PCD (col. 3, II. 31-35) 
as a system time. Maturi teaches the clock as a counter (col. 7, II. 22-27). Maturi 
teaches the STC derived from SCR or PCR (col. 3, II. 31-43), which is divided to reduce 
the resolution. Maturi teaches a comparator (fig. 10, label 42, fig. 11, label 42), and the 
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pre-parser (22) and post-parser (24) clearly have registers in order to process the audio 
decoding in less than one frame time interval different from the counter (col. 7, II. 37- 
40). As described previously in the rejection, there exist PTS and presentation units 
(PU) which are not necessarily sent together, which equate to the first, second, and 
third data unit presentation moments (col. 1-2, II. 66-4), which are clearly depacketized 
in order to retrieve the data. The system of Maturi provides the time recovery unit and 
the presentation units (fig. 3). 

Maturi teaches the events being audio and video, but is silent on an event apart 
from reconstructing video and audio. In analogous art, Srinivasan teaches integrating 
an event, such as host spots and interactive regions (col. 37, II. 3-6) in order to present 
metadata during a frame or series of frames (col. 37, II. 35-41 ). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to modify Maturi by synchronizing events another than audio and video as taught by 
Srinivasan in order to enhance the main video stream an incorporating metadata 
thereby increasing interactivity and enabling the user to select and review portions of 
video presentations (Srinivasan: col. 3, II. 8-23, col. 3, II. 42-52). 

Maturi teaches generating a system and application time (see discussion of claim 
9) and selecting a PTS (claimed sample moment of the application time) at the encoder 
(col. 1, II. 62-65) and sending the data downstream to the user in a first data unit (col. 1- 
2, II. 66-4). Further, Maturi teaches generating a DTS, synchronized to the system time 
by the sample moment and first data unit presentation moment (col. 7, II. 34-37), further 
the examiner notes that by determining the DTS, the DTS is associated with the audio 
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even and said instant of said video with a correlating moment of said application time, 
such as DTS = PTS - fixed decoding time; Maturi teaches presenting the system time in 
order to play the audio event synchronized with the video (col. 7, II. 22-40). In addition, 
fig 3 has a pre-parser 22 and a post-parser 24. 



Conclusion 

6. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .1 36(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrew Y Koenig whose telephone number is (703) 
306-0399. The examiner can normally be reached on M-Th (7:30 - 6:30). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Grant can be reached on (703) 305-4755. The fax phone 
number for the organization where this application or proceeding is assigned is 703- 



Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



872-9306. 



ayk 




CHRIS GRANT 
PRIMARY EXAMINER 



